Skip to content

benchmarks.yml: Use prebuilt tools for reproducible benches#10497

Merged
sylvestre merged 2 commits intouutils:mainfrom
oech3:patch-14
Feb 16, 2026
Merged

benchmarks.yml: Use prebuilt tools for reproducible benches#10497
sylvestre merged 2 commits intouutils:mainfrom
oech3:patch-14

Conversation

@oech3
Copy link
Contributor

@oech3 oech3 commented Jan 26, 2026

No description provided.

@github-actions
Copy link

github-actions bot commented Jan 26, 2026

GNU testsuite comparison:

Congrats! The gnu test tests/date/date-ethiopia is no longer failing!
Congrats! The gnu test tests/date/date-iran is no longer failing!
Congrats! The gnu test tests/date/date-thailand is no longer failing!

@oech3 oech3 force-pushed the patch-14 branch 3 times, most recently from b08f09d to 2b7b00c Compare January 26, 2026 19:36
@oech3
Copy link
Contributor Author

oech3 commented Jan 26, 2026

Hmm?

@oech3 oech3 force-pushed the patch-14 branch 2 times, most recently from df24cfb to f3f0e6b Compare January 26, 2026 19:45
@oech3 oech3 marked this pull request as ready for review January 26, 2026 19:45
@codspeed-hq
Copy link

codspeed-hq bot commented Jan 26, 2026 •
edited
Loading

Merging this PR will not alter performance

✅ 284 untouched benchmarks
⏩ 38 skipped benchmarks1

Comparing oech3:patch-14 (ab3aa69) with main (ec7e81e)

Open in CodSpeed

Footnotes

  1. 38 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@github-actions
Copy link

github-actions bot commented Jan 26, 2026

GNU testsuite comparison:

Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/timeout/timeout (passes in this run but fails in the 'main' branch)

@github-actions
Copy link

github-actions bot commented Jan 27, 2026

GNU testsuite comparison:

GNU test failed: tests/timeout/timeout-group. tests/timeout/timeout-group is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/tail/follow-name (fails in this run but passes in the 'main' branch)

@oech3
Copy link
Contributor Author

oech3 commented Jan 27, 2026

@xtqqczze Should we keep cargo install cargo-codspeed for stable benchmarks?

@oech3
Copy link
Contributor Author

oech3 commented Jan 27, 2026

caused by preinstalled rust?

xtqqczze
xtqqczze reviewed Jan 27, 2026
View reviewed changes
xtqqczze
xtqqczze reviewed Jan 27, 2026
View reviewed changes
@oech3 oech3 mentioned this pull request Jan 27, 2026
Open
@xtqqczze
Copy link
Contributor

xtqqczze commented Jan 27, 2026

@xtqqczze Should we keep cargo install cargo-codspeed for stable benchmarks?

It is recommended to use binary install in CI:

https://codspeed.io/docs/reference/codspeed-rust/cargo-codspeed#with-cargo-binstall-recommended-in-ci

So taiki-e/install-action is best here.

@oech3

This comment was marked as resolved.

Sign in to view
@oech3 @xtqqczze
benchmarks.yml: Use prebuilt tools for faster setup
caf4c45 
Co-authored-by: xtqqczze <45661989+xtqqczze@users.noreply.github.com>
@oech3
Copy link
Contributor Author

oech3 commented Jan 27, 2026

Yes. We should same binary for benchmark producing same result.

@oech3 oech3 changed the title benchmarks.yml: Use prebuilt tools for faster setup benchmarks.yml: Use prebuilt tools for reproducible benches Jan 27, 2026
@github-actions
Copy link

github-actions bot commented Jan 27, 2026

GNU testsuite comparison:

Skipping an intermittent issue tests/shuf/shuf-reservoir (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/sort/sort-stale-thread-mem (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/printf/printf-surprise is now being skipped but was previously passing.
Congrats! The gnu test tests/env/env-signal-handler is now passing!

@xtqqczze
Copy link
Contributor

xtqqczze commented Jan 27, 2026 •
edited
Loading 

info: host platform: x86_64_linux
info: install-action does not support cargo-codspeed; fallback to cargo-binstall
info: installing cargo-binstall@latest (1.17.3)
info: downloading https://github.com/cargo-bins/cargo-binstall/releases/download/v1.17.3/cargo-binstall-x86_64-unknown-linux-musl.tgz
info: verifying sha256 checksum for cargo-binstall-x86_64-unknown-linux-musl.tgz

Why is taiki-e/install-action installing musl binaries, when cargo-binstall is available for x86_64-unknown-linux-gnu?

@oech3
Copy link
Contributor Author

oech3 commented Jan 27, 2026

Improved by some environment var?

@xtqqczze
Copy link
Contributor

xtqqczze commented Jan 27, 2026

https://github.com/taiki-e/install-action/blob/afd4ea001d8f1c0a9fd9a77a86ef6deebc6a621a/main.sh#L259-L261

@oech3
Copy link
Contributor Author

oech3 commented Jan 27, 2026

Just follow with action's default. It works. No need to optimize toolchain's performance.

@github-actions
Copy link

github-actions bot commented Feb 11, 2026

GNU testsuite comparison:

Congrats! The gnu test tests/pr/bounded-memory is no longer failing!

@sylvestre sylvestre merged commit bd58575 into uutils:main Feb 16, 2026
154 of 155 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@xtqqczze xtqqczze xtqqczze left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@oech3 @xtqqczze @sylvestre